Learning based modeling of emergent behaviour of complex system

ABSTRACT

The disclosure generally related to a learning-based modelling of an emergent behavior of a complex system. Existing decision-making at complex systems primarily relies on qualitative approaches, which often results in inaccurate outputs. The disclosed system includes a digital twin of the complex system and a digital twin of an environment of said complex system, and captures an interaction and dynamic behavior of agents of the digital twins. The agents of the digital twins are simulated and modelled using learning-based models such as RL and genetic algorithms that learns the behavior (i.e. actions and their outcomes) over a period of time. Hence, the agents (or actors) of the digital twins are dynamic in nature. The actor-based bottom up simulation approach is capable of producing sufficient insight for effective decision making prior to implementation.

PRIORITY CLAIM

This U.S. patent application claims priority under 35 U.S.C. § 119 to:India Application No. 202121008298, filed on Feb. 26, 2021. The entirecontents of the aforementioned application are incorporated herein byreference.

TECHNICAL FIELD

The disclosure herein generally relates to complex systems, and, moreparticularly, to learning based modeling of emergent behavior of complexsystems.

BACKGROUND

Complex systems, for instance, modern enterprises are systems withpeople process and IT infrastructure. Each system has independentexistence and goals. They operate in a dynamic uncertain environment andcontinuously evaluate their status-quo and evolve to stay competitiveand economically viable in the current business environment. In thisendeavor, decision-makers constantly explore the answers for a range ofdecision questions such as: Is the current form of the organizationappropriate to stay ahead of competition or economically viable? Whatkind of changes are necessary to achieve organizational goals? Where toapply those change? When to apply those changes?

Addressing these decision questions requires understanding of multipleaspects of the enterprise and its operating environment. This isextremely difficult due of the characteristics of the enterprise thatincludes socio-technical aspects, organizational structure, inherentuncertainty, and emergent behavior, among others.

Existing decision-making at complex systems primarily relies onqualitative approaches, such as discussion and interviews, with limitedquantitative assistance that comes from spreadsheets-based datacomputation. There exists excessive dependency on human intuitions andinterpretations. These often results in a less effective decision. Thisis especially true when the context is complex, dynamic, and uncertain.

There exists multiple enterprise modelling and analysis techniquessupporting quantitative approaches for organizational decision making.However, their utility is limited to a class of decision-making problemscompared to a wide range of problems. For example, inferentialtechniques that rely on the statistical interpretation of historicalsystem data are suitable only for static environments. The mathematicalmodels, such as linear programming, work well for mechanistic andmonolithic systems that are not adaptive in nature. The enterprisemodels, such as ArchiMate, i*, and BPMN, are found to be inappropriatefor the systems that exhibit significant uncertainty and emergentbehavior. The actor technologies and agent-based systems fall short ofexpressing the complex organizational structure and uncertainty. Thedecision-making challenges may be dealt with in the enterprise utilizingmodelling and analysis techniques.

SUMMARY

Embodiments of the present disclosure present technological improvementsas solutions to one or more of the above-mentioned technical problemsrecognized by the inventors in conventional systems. For example, in oneembodiment, a method for learning based modelling of complex system isprovided. The method includes simulating, via one or more hardwareprocessors, a first digital twin of a complex system and a seconddigital twin of an environment associated with the complex system, thefirst digital twin comprising a first set of digitally configureddynamic agents and the second digital twin comprising a second set ofdigitally configured dynamic agents, each of the first set and thesecond set of digitally configured dynamic agents defined using one ormore state variables, one or more characteristic variables and a set ofactions. Further the method includes receiving via the one or morehardware processors, a trigger at one or more digitally configureddynamic agents from amongst the first set and the second set ofdigitally configured dynamic agents. Furthermore, the method includescomputing, via the one or more hardware processors, a current value ofthe one or more state variables and the one or more characteristicvariables associated with the one or more digitally configured dynamicagents by accessing a system database. Moreover, the method includes,predicting, via the one or more hardware processors, a differencebetween the current value and an expected value of the one or more statevariables and the one or more characteristics variables, the currentvalue computed using multi criteria decision making technique, theexpected value obtained from one or more goals associated with thecomplex system, the one or more goals prestored in a knowledgerepository associated with the complex system, and wherein the one ormore goals are indicative of decision-making in response to the triggerin the complex system. Also, the method include defining, based on thedifference between the current value and the expected value of the oneor more state variables and the one or more characteristic variables, adecision function for the one or more digitally configured dynamicagents using a decision function, via the one or more hardwareprocessors, wherein the decision function is realized using at least oneof a Reinforcement learning (RL) and optimization technique, and whereinthe observation is indicative of outcome of the decision function andability to reach to the desired state in the future time. The first andsecond set of digitally configured dynamic agents are simulatediteratively via the one or more hardware processors, based on thedecision function in a plurality of iterations until the differencebetween the current value and expected value of the one or more statevariables is determined to be within a predetermined threshold limit.

In another aspect, a system for learning based modelling of complexsystem is provided. The system includes a memory storing instructions;one or more communication interfaces; and one or more hardwareprocessors coupled to the memory via the one or more communicationinterfaces, wherein the one or more hardware processors are configuredby the instructions to simulate a first digital twin of a complex systemand a second digital twin of an environment associated with the complexsystem, the first digital twin comprising a first set of digitallyconfigured dynamic agents and the second digital twin comprising asecond set of digitally configured dynamic agents, each of the first setand the second set of digitally configured dynamic agents defined usingone or more state variables, one or more characteristic variables and aset of actions. Further, the one or more hardware processors areconfigured by the instructions to receive a trigger at one or moredigitally configured dynamic agents from amongst the first set and thesecond set of digitally configured dynamic agents. Furthermore, thecompute a current value of the one or more state variables and the oneor more characteristic variables associated with the one or moredigitally configured dynamic agents by accessing a system database.Also, the one or more hardware processors are configured by theinstructions to predict a difference between the current value and anexpected value of the one or more state variables and the one or morecharacteristics variables, the current value computed using multicriteria decision making technique, the expected value obtained from oneor more goals associated with the complex system, the one or more goalsprestored in a knowledge repository associated with the complex system,and wherein the one or more goals are indicative of decision-making inresponse to the trigger in the complex system. Moreover, the one or morehardware processors are configured by the instructions to define, basedon the difference between the current value and the expected value ofthe one or more state variables and the one or more characteristicvariables, a decision function for the one or more digitally configureddynamic agents using a decision function, wherein the decision functionis realized using at least one of a Reinforcement learning (RL) andoptimization technique, and wherein the observation is indicative ofoutcome of the decision function and ability to reach to the desiredstate in the future time. The one or more hardware processors arefurther configured by the instructions to simulate iteratively the firstand second set of digitally configured dynamic agents based on thedecision function in a plurality of iterations until the differencebetween the current value and expected value of the one or more statevariables is determined to be within a predetermined threshold limit.

In yet another aspect, a non-transitory computer readable medium for amethod for learning based modelling of complex system is provided. Themethod includes simulating, via one or more hardware processors, a firstdigital twin of a complex system and a second digital twin of anenvironment associated with the complex system, the first digital twincomprising a first set of digitally configured dynamic agents and thesecond digital twin comprising a second set of digitally configureddynamic agents, each of the first set and the second set of digitallyconfigured dynamic agents defined using one or more state variables, oneor more characteristic variables and a set of actions. Further themethod includes receiving via the one or more hardware processors, atrigger at one or more digitally configured dynamic agents from amongstthe first set and the second set of digitally configured dynamic agents.Furthermore, the method includes computing, via the one or more hardwareprocessors, a current value of the one or more state variables and theone or more characteristic variables associated with the one or moredigitally configured dynamic agents by accessing a system database.Moreover, the method includes, predicting, via the one or more hardwareprocessors, a difference between the current value and an expected valueof the one or more state variables and the one or more characteristicsvariables, the current value computed using multi criteria decisionmaking technique, the expected value obtained from one or more goalsassociated with the complex system, the one or more goals prestored in aknowledge repository associated with the complex system, and wherein theone or more goals are indicative of decision-making in response to thetrigger in the complex system. Also, the method include defining, basedon the difference between the current value and the expected value ofthe one or more state variables and the one or more characteristicvariables, a decision function for the one or more digitally configureddynamic agents using a decision function, via the one or more hardwareprocessors, wherein the decision function is realized using at least oneof a Reinforcement learning (RL) and optimization technique, and whereinthe observation is indicative of outcome of the decision function andability to reach to the desired state in the future time. The first andsecond set of digitally configured dynamic agents are simulatediteratively via the one or more hardware processors, based on thedecision function in a plurality of iterations until the differencebetween the current value and expected value of the one or more statevariables is determined to be within a predetermined threshold limit.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory onlyand are not restrictive of the invention, as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of this disclosure, illustrate exemplary embodiments and, togetherwith the description, serve to explain the disclosed principles:

FIG. 1 illustrates typical core concepts of decision-making in a complexsystem using a meta-model.

FIG. 2A describes a first block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2B describes a second block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2C describes a third block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2D describes a fourth block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2E describes a fifth block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2F describes a sixth block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2G describes a seventh block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2H describes an eight block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2I describes a ninth block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 2J describes a tenth block diagram of a method and system forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 3 illustrates an environment and complex system topology forlearning based modelling of a complex system in accordance with exampleembodiments of the present disclosure.

FIG. 4 illustrates a flow chart of a method for learning based modellingof a complex system in accordance with example embodiments of thepresent disclosure.

FIG. 5 illustrates structure of a digitally configured dynamic agent inaccordance with an example embodiment.

FIG. 6 illustrates a flow diagram of the method for learning basedmodelling of a complex system in accordance with example embodiments ofthe present disclosure.

FIG. 7 is a block diagram of an exemplary computer system forimplementing embodiments consistent with the present disclosure

DETAILED DESCRIPTION

Two critical aspects of organizational decision-making includes (1) Whatand how to capture the necessary information of an organizationaldecision-making problem, and (2) How to analyze various decisionalternatives and understand their consequences prior to theirimplementation.

Organizational theory and management studies on decision-making help inidentifying the necessary information of an organizationaldecision-making (i.e. what to capture). Once the necessary informationbecomes available, systematically capturing and utilizing thisinformation to analyze multiple decision choices and understanding theirconsequences prior to their implementation is vitally important.Effective decision-making based on precise understanding of anorganization is critical for modern organizations to stay competitive ina dynamic and uncertain business environment.

Management theories describe decision-making using three broad concepts,namely: decision problem, course of action and decision. The decisionproblem is organizational goals that an organization targets, courses ofaction is the knowledge of alternatives that are considered andevaluated in a decision-making action, and a decision is the outcome ofa decision-making action, i.e., selected alternative. However, thestate-of-the-art technologies that are relevant in this context are notadequate to capture and quantitatively analyze complex organizations.

It requires specific contextual information to evaluate the consequencesof potential courses of action, i.e., develop knowledge of consequences.Methodologically, a decision-making is approached using four steps,namely (1) problem identification, i.e., defining precise decisionproblem (2) generation of alternative courses of action, i.e.,development of knowledge of alternatives for a decision problem, (3)evaluation of courses of action or developing knowledge of consequencesby predicting/computing the key performance indicators (KPIs) fromcontextual information, and (4) ranking of courses of action (i.e.,consequent preference ordering) and selection of the most effectivecourse of action (i.e. a decision).

The embodiments disclosed herein provides a system and method to developthe system to enable the decision makers in evaluating differentdecision alternatives on a digital representation of the real enterpriseand to help them identify the optimal choices to be applied on the realenterprise in a reliable and automated manner. For example, thedisclosed system is capable of capturing the necessary information oforganizational/complex system's decision-making effectively. The systemis further capable of analyzing what-if scenarios. In an embodiment, thedisclosed system captures the necessary information utilizing thefollowing two concepts: (1) system of systems and (2) an actor model ofcomputation. The aforementioned concepts lead to the creation of adigital replica of the complex system, also known as a digital twin. Thedigital twin captures the structure as well as the behavior of thecomplex system. The actor model enables the digital twin to replicatethe complex system in a bottom-up fashion. It also allows to performsimulation of different aspects of the complex system on its digitaltwin. The analysis approach utilizes the bottom-up simulation techniqueto understand the key characteristics of complex system such as:autonomy, adaptability, uncertainty, and emergent behavior.

Exemplary embodiments are described with reference to the accompanyingdrawings. In the figures, the left-most digit(s) of a reference numberidentifies the figure in which the reference number first appears.Wherever convenient, the same reference numbers are used throughout thedrawings to refer to the same or like parts. While examples and featuresof disclosed principles are described herein, modifications,adaptations, and other implementations are possible without departingfrom the scope of the disclosed embodiments. It is intended that thefollowing detailed description be considered as exemplary only, with thetrue scope being indicated by the following claims.

Referring now to the drawings, and more particularly to FIG. 1 through7, where similar reference characters denote corresponding featuresconsistently throughout the figures, there are shown preferredembodiments and these embodiments are described in the context of thefollowing exemplary system and/or method.

FIG. 1 illustrates typical core concepts of organizationaldecision-making using a meta-model 100. The concept of decision-makingare represented using three entities: Goal, Measures, and Lever. Theconcept Goal represents the organizational goals. Measure represents thekey performance indicators (KPIs) that indicate the fulfillment ofGoals. A Lever is a conceptual representation of a course of action.

The contextual information for decision-making is represented using twoprimitive elements: Organization and Environment. An organization isvisualized as a system that has Structure, Behavior and State. Moreover,an organization often records its historical states, interactions,realization of goals, and the useful phenomena as an organizationalmemory. These historical records are termed as Trace. Operationally,Behavior updates the State and Trace of an organization. The concepts ofGM-L (Goals, Measure, Lever) structure and contextual informationconverge at two concepts: Lever and Measure. A Lever of a GM-L structuredescribes the changes of Organization elements that include Structure,Behavior, and Goal; whereas, the Measures are expression over Trace andState. In this formulation, an organizational decision making is amethod to develop the knowledge of consequences by computing/predictingthe Measures for all identified Levers (i.e. the knowledge ofalternatives), rank the Levers based on the observed Measure values(i.e., consequent preference ordering), and select a Lever that servesthe purpose best (i.e., decision).

Typically, a complex system is a reactive entity (as it exchangesmessages and resources with its environment). The complex entity oftencomposes a large number of interdependent subsystems or elements (i.e.,system of systems) in a nonlinear way. The complex system may becharacterized by a composition of multiple loosely coupled andautonomous elements. The behavior of a complex organization is largelyprobabilistic and emerges from the interactions of the connectedsub-systems and individuals. Accordingly, a complex system may bevisualized as a system of systems, where each constituent system ischaracterized by multiple socio-technical properties such as:modularity, composability, autonomy, temporality, reactiveness,adaptability, uncertainty and emergentism.

The disclosed embodiments present method and system to facilitatemodelling and analysis technique for supporting quantitativeevidence-driven decision-making in the complex system. For example, thedisclosed embodiments present system and method for decision-makingprocess using a digital twin of the complex system and a digital twin ofthe environment of the complex system. The method and system aredescribed further with reference to FIGS. 2A-7.

Referring collectively to FIGS. 2A-6, a method for learning basedmodelling of a complex system in accordance with example embodiments ofthe present disclosure. For example, FIGS. 2A-2F discloses method andsystem for learning based modelling of a complex system in accordancewith example embodiments of the present disclosure. FIG. 3 illustrates aflow chart of a method for learning based modelling of a complex systemin accordance with example embodiments of the present disclosure. FIG. 4illustrates an environment and complex system topology for learningbased modelling of a complex system in accordance with exampleembodiments of the present disclosure. FIG. 5 illustrates structure of adigitally configured dynamic agent in accordance with an exampleembodiment. FIG. 6 illustrates a flow diagram of the method for learningbased modelling of a complex system in accordance with exampleembodiments of the present disclosure.

Herein, for the purpose of explanation, an enterprise (or anorganization) is taken as an example of the complex system, however, itwill be understood that the any complex system which includes a systemsand subsystems may be defined as a complex system, such as a city, astate, a country, an educational institute, a training institute, ane-commerce system, and so on and so forth.

FIG. 2A illustrates a digital twin for decision making in accordancewith an example embodiment. FIG. 2B represents steps in thedecision-making process using a digital twin in accordance with anexample embodiment. As is seen with reference to FIGS. 2A-2B, once theneed for the digital twin and the corresponding goals and measures areidentified for a decision-making problem, the digital twin isconstructed. The digital twin represents the structure and behavior ofthe enterprise and it is constructed using actor model of computation.The construction process involves identification of the relevantinformation from various information sources from the enterprise. Someof the sources are organizational objectives, vision, reports,structure, processes, and existing IT systems, among others. Theconstruction process is semi-automatic and involves subject matterexperts or domain experts.

An actor meta-model used to construct a digital twin of the complexsystem for simulation is already described with reference to FIG. 1.FIG. 2C illustrates an example typical representation of theorganization's (or complex system's) meta-model for decision making. Theorganization meta model highlights goal, measure and lever entities andtheir relations to relevant entities for organizational decision making.A block diagram broadly describing the decision-making steps at a systemfor the complex system using the digital twin is shown in FIG. 2D.

As illustrated in FIG. 2D, the block diagram of a system for decisionmaking of the complex system includes a modeler component, a validatorcomponent, a synchronizer component, a simulator component, a visualizercomponent, and a recommender component. The aforementioned components ofthe system are described further in detail below with reference to FIGS.2E-2I.

Referring collectively to FIGS. 2E-2I, once the digital twin isconstructed, the validation process is carried out to ensure faithfulrepresentation of the real enterprise (or the complex system). Varioustechniques from operational validation may be employed. Historical datafrom different information technology (IT) systems of the complex systemmay be utilized and the digital twin is subjected to past events(obtained from the historical data) to simulate the past behavior. Ifthis simulation leads to observations that tally with real historicalobservations, the validation process is successful. Otherwise, theconstruction process is revisited, gaps between the real system and thedigital twin are identified, and the digital twin is suitably modified.The modified twin is again subjected to validation.

As illustrated in FIG. 2A, the digital twin (for example, the firstdigital twin and the second digital twin) for learning based modeling ofemergent behavior of complex systems may be constructed as a faithfulrepresentation of the real system and instantiated with real data fromthe complex system. Decision making experiments and explorations may becarried out on the digital twin using the bottom-up simulationtechniques, and the generated data or observations from the digital twinenables selection of optimal choices for decision making.

Next, the digital twin is initialized or instantiated with data fromvarious information sources from the enterprise. The primary sources ofthe data used for synchronization comes from existing IT systems of theenterprise. Extract-Transform-Load (ETL) process may be employed forthis purpose. Some of the information required for initialization of thedigital twin may not be available in existing systems directly. Variousdata analytics or machine learning techniques may be employed to computethe missing information. After initialization, the digital twin is readyfor simulation.

During simulation (as shown in FIG. 2H), the digital twin may besubjected to all the events that are expected in the real system incertain period. During the simulation process, the state of the digitaltwin undergoes changes in response to the events. The identifiedmeasures are computed. The information generated during simulation isstored for further analysis and interpretation.

The information generated during simulation are analyzed and interpretedby different stakeholders such as domain experts and decision makers.Different stake holders require different views on the information andthe information is presented through various visualization aids (asillustrated in FIG. 2I). For example, domain experts may be interestedin changes in customer base while decision makers may be interested inoverall revenue or profit.

The final step in the overall process is recommendation or actualdecision making which is based on outcome of the digital twin. If thecomputed measures for the identified goals are acceptable to all thestakeholders, levers (if any) are deemed appropriate and the suitablerecommendation are suggested for the real system. Otherwise, differentwhat-if scenarios are tried out by adopting or adjusting various leversand their outcomes are evaluated. Various decision-making techniques areadopted to optimize the levers to achieve the goals as indicated by themeasures. Some of the techniques adopted are reinforcement learning,genetic algorithm, linear programming, etc.

The very first simulation of the digital twin is performed withoutadopting any levers to measure the as-is state of the enterprise. If themeasures indicate the goals are achieved without introducing any levers,the current state of the real enterprise (people, processes, andsystems) are deemed appropriate and no modification or changes arerecommended. On the contrary, the measures may indicate the goals arenot achieved with as-is system. This indicates there is a need forupdate and various if-what scenarios are simulated by adopting variouslevers. Multiple iterations through all the steps may be required toachieve the stated goals. Decision enablers such as reinforcementlearning (RL) or genetic algorithm (GA) helps in reducing the number ofiterations.

Aforementioned FIGS. 2E-2I described system components for simulating adigital twin. In accordance with various embodiments, the complex system(e.g. an enterprise) and environment of the complex system configures atopology as set of interacting agents as shown in FIG. 3. The complexsystem may be associated with certain traits, including, for example,it's a system of systems. The complex system may include multiple unitswhich may be intentional, autonomous, reactive and probabilistic. Theunits may exhibit spatio-temporal characteristics and non-linearinteractions, in light of which, the overall behavior of the complexsystem may emerge with time as described above. The disclosed system iscapable of observing the set of actions of the complex system and itsenvironment to capture emergent behavior of the complex system.

As previously described, the emergent behavior of the complex system canbe modeled by using a digital twin which further may be utilized forpredictions in decision-making process. A first digital twin of thecomplex system and a second digital twin of the environment associatedwith the complex system are simulated at 402. In an embodiment, thefirst digital twin and the second digital twin may be simulated asdescribed with reference to FIGS. 2D-2I.

The first digital twin includes a first set of digitally configureddynamic agents and the second digital twin includes a second set ofdigitally configured dynamic agents. The first set and the second set ofdigitally configured dynamic agents may hereinafter be collectivelyreferred to as agents. The agents are capable of learning over a periodof time by observing the changes (or the set of actions) in the complexsystem and the environment. In this context, the characteristics of anagent change over time by observing some or many (or a pattern of)actions resulting into good and/or bad state over the time. For both thescenarios, the agents may change characteristics thereof based onobservation (of actions and results of actions), and hence, the agentsare termed as digitally configured dynamic agents. The structure of anagent is described with reference to FIG. 5.

Referring to FIG. 5, a schematic representation of a digitallyconfigured dynamic agent, for example, each agent of the first set andthe second set of the digitally configured dynamic agents is presentedin accordance with an example embodiment. As illustrated, each agent isformed using a tuple: <Goal, state variables, characteristics variables,a set of Actions>, where an action is defined as tuple <Event, Trigger,Computation, Resistance>. Each of the first set and the second set ofthe digitally configured dynamic agents may be defined using one or morestate variables, one or more characteristic variables and a set ofactions.

For the ease of understanding, the disclosed embodiments are explainedby taking a simple example of a telecom customer as the agent (refer toFIG. 5). The customer may assume various states that may be representedby way of state variables. For example, in the present scenario, if thetelecom customer has not paid last month's bill, then the customer mayassume the state as ‘defaulter’. The one of more characteristicvariables determine the characteristics and/or behavioral pattern orbehavioral traits of an customer (or the agent). For example,characteristics of a telecon customer may be—(a) ‘delayed payer’—paybill with fine after specified last date, (b) ‘frequent caller’—callcustomer care for every small network issues).

In an example scenario, the customer may wish to take a phone plan andmay have two options, namely Plan A and Plan B. Plan A may be [price399, 100 calls/day and 1 GB data/day] and Plan B may be [price 499, 50calls/day and 4 GB data/day). The telecom customer may select a planbased on the need (i.e., State represented as state variable) andcharacteristics (represented as characteristic variables). If thecustomer is not frequent caller and not a price sensitive person, thenthe customer may select Plan B otherwise the customer may select Plan A.Herein, it will be noted that the aforementioned example is taken forthe ease of understanding of the embodiments and to illustrate themeanings of terms, however, in case of complicated examples (offeringhuge number of options and attributes) the selection process may becomecomplicated.

In an embodiment, the one or more state variables describe the currentstate of an agent (e.g. S1, S2, S_(n) illustrated in FIG. 5). The one ofmore characteristic variables determine the characteristics orbehavioral pattern or behavioral traits of the agent (D₁, D₂, . . .D_(n) in FIG. 5).

An event is a meaningful phenomenon where a phenomenon is commutatedusing pattern matching on predefined activities occurring within agent,enterprise (i.e. the complex system) and/or environment. The event maybe a specific happening outside of an agent. For example, in theaforementioned example, an event for a customer (or agent) may be—a)receiving an short messaging service (SMS) stating telecom company agenthave introduced a new product or offering, b) receiving a bill, c)receiving a news that all other customers are happy/unhappy aboutcertain product, and so on and so forth.

A trigger is true iff (Event) occurs and (Expression on (CharacteristicsVariables)==True)

Computation refers to a ComputationFunction (State Variables,Characteristics Variables), where multicriteria decision-making (MCDC)is used as an option for ComputationFunction.

Resistance value function refers to a threshold of an agent to take aspecific action. For example, a customer (i.e. agent in this case) mightknow that a product is better suited for his/her needs but may not besignificant value-add to switch to said product. This function dependson the characteristic variables (for example, customer profile includingbehavioral traits) of the customer. Herein the ‘swithing to a differentproduct’ is the ‘event’.

Resistance: ComputedValue−Threshold, where Threshold is a function overCharacteristics Variables.

It will be noted that an action can be triggered by an agent whenoutcome of computation function is greater than the threshold value (orresistance value), which is a function of characteristic variables. Eachagent may have different threshold to act/trigger an action.

At 404, the method 400 includes receiving a trigger event at one or moredigitally configured dynamic agents from amongst the first set and thesecond set of digitally configured dynamic agents.

At 406, the method 400 includes computing a current value of the one ormore state variables and the one or more characteristic variablesassociated with the one or more digitally configured dynamic agents byaccessing a system database. In an embodiment, the current value of thestate variables and the characteristic variables are obtained usingcomputed using multi criteria decision making (MCDM) technique. MCDMranks a set of options from the attributes of the options andcorresponding affinities of the attributes. For instance, in theaforementioned example, phone plans (Plan A and Plan B) may be the setof options available, and the need of the customer may define the statevariables, and the behavior or nature of the customer may definecharacteristics variables.

At 408, the method 400 includes predicting a difference between thecurrent value and an expected value of the one or more state variablesand the one or more characteristics variables.

Difference=KPIs (current values of the state variables)−Goal (expectedvalues of the state variables).

The expected value of the state variables may be obtained from one ormore goals associated with the complex system. The one or more goals maybe prestored in a knowledge repository associated with the complexsystem. For example, in case of an enterprise, the knowledge repositorymay be a system database associated with the enterprise. The one or moregoals are indicative of decision-making in response to the trigger inthe complex system.

Observation=polarity of the ‘Difference’ (i.e., if Difference>=0 then itis positive observation else negative observation)

At 410, the method 400 includes defining, based on a difference betweenthe current value and the expected value of the one or more statevariables and the one or more characteristic variables, a decisionfunction for the one or more digitally configured dynamic agents usingdecision function. In an embodiment, the decision function may berealized using Reinforcement learning (RL) and/or optimizationtechnique. In an embodiment, each agent may include a reinforcementlearning agent that is capable of observing actions taken by thedecision function and consequence of the action (i.e., said action ishelping to reach desired state variable values) during the iterativesimulations (i.e., predicting if agent is reaching its goal in thefuture or not). Herein, an observation is indicative of outcome of thedecision function and ability to reach to a desired state in a futuretime. For example in the aforementioned example, if an agent observes(as defined using term ‘observation’) its selection of a Plan (e.g.,Plan A or Plan B) resulting into unsatisfactory results (i.e. negativeobservation) for a predefined number of observations, then the agent maychange characteristics variables (e.g., price sensitive to non-pricesensitive) thereof using a learning technique (for example reinforcementlearning) so as to get positive observations. This type ofself-adaptation may be triggered after a specific learning over multipleobservations (i.e. experience).

At 412, the method 400 includes simulating iteratively the first andsecond set of digitally configured dynamic agents based on the decisionfunction in a plurality of iterations until the difference between thecurrent value and expected value of the one or more state variables isdetermined to be within a predetermined threshold limit.

In an embodiment, the actions taken by each agent over a predefinedperiod of time may lead to learning of that agent. For example, based ona series of actions taken by the digitally configured dynamic agent overa predefined period of time and observing values in terms of the one ormore state variables in cognizance of the one or more goals of thedigitally configured dynamic agent and actual state variable after thepredefined period of time, the system may trigger a modification in theone or more characteristics variables of the agent. For example, for(all desired state variable)

Value_(state_variable)=desired state variable−actual state variable

Overall Value=Function (All Value_(state_variables)), where Function canbe specified using MCDM technique as some Value_(state_variable) may bepositive and other may be negative with different magnitude.

-   -   If (Overall Value>0 or a threshold value) then combination of        characteristics variables is assumed to help the agent in a        positive direction so no change in characteristic variable is        required but if the overall value is below the threshold, then        the agent can adapt to achieve the goal by changing        characteristic variables. In an embodiment, RL is used to decide        the characteristic variables that should be changed and the        change to be introduced in such characteristic variables of the        agent.

FIG. 7 is a block diagram of an exemplary computer system 701 forimplementing embodiments consistent with the present disclosure.Variations of computer system 701 may be used for implementing thedevices included in this disclosure. Computer system 701 may comprise acentral processing unit (“CPU” or “hardware processor”) 702. Thehardware processor 702 may comprise at least one data processor forexecuting program components for executing user- or system-generatedrequests. The processor may include specialized processing units such asintegrated system (bus) controllers, memory management control units,floating point units, graphics processing units, digital signalprocessing units, etc. The processor may include a microprocessor, suchas AMD Athlon™, Duron™ or Opteron™ ARM's application, embedded or secureprocessors, IBM PowerPC™, Intel's Core, Itanium™, Xeon™, Celeron™ orother line of processors, etc. The processor 702 may be implementedusing mainframe, distributed processor, multi-core, parallel, grid, orother architectures. Some embodiments may utilize embedded technologieslike application specific integrated circuits (ASICs), digital signalprocessors (DSPs), Field Programmable Gate Arrays (FPGAs), etc.

Processor 702 may be disposed in communication with one or moreinput/output (I/O) devices via I/O interface 703. The I/O interface 703may employ communication protocols/methods such as, without limitation,audio, analog, digital, monoaural, RCA, stereo, IEEE-1394, serial bus,universal serial bus (USB), infrared, PS/2, BNC, coaxial, component,composite, digital visual interface (DVI), high-definition multimediainterface (HDMI), RF antennas, S-Video, VGA, IEEE 802.11 a/b/g/n/x,Bluetooth, cellular (e.g., code-division multiple access (CDMA),high-speed packet access (HSPA+), global system for mobilecommunications (GSM), long-term evolution (LTE), WiMax, or the like),etc.

Using the I/O interface 703, the computer system 701 may communicatewith one or more I/O devices. For example, the input device 704 may bean antenna, keyboard, mouse, joystick, (infrared) remote control,camera, card reader, fax machine, dongle, biometric reader, microphone,touch screen, touchpad, trackball, sensor (e.g., accelerometer, lightsensor, GPS, gyroscope, proximity sensor, or the like), stylus, scanner,storage device, transceiver, video device/source, visors, etc.

Output device 705 may be a printer, fax machine, video display (e.g.,cathode ray tube (CRT), liquid crystal display (LCD), light-emittingdiode (LED), plasma, or the like), audio speaker, etc. In someembodiments, a transceiver 706 may be disposed in connection with theprocessor 702. The transceiver may facilitate various types of wirelesstransmission or reception. For example, the transceiver may include anantenna operatively connected to a transceiver chip (e.g., TexasInstruments WiLink WL1283, Broadcom BCM4750IUB8, Infineon TechnologiesX-Gold 618-PMB9800, or the like), providing IEEE 802.11a/b/g/n,Bluetooth, FM, global positioning system (GPS), 2G/3G HSDPA/HSUPAcommunications, etc.

In some embodiments, the processor 702 may be disposed in communicationwith a communication network 708 via a network interface 707. Thenetwork interface 707 may communicate with the communication network708. The network interface may employ connection protocols including,without limitation, direct connect, Ethernet (e.g., twisted pair10/100/1000 Base T), transmission control protocol/internet protocol(TCP/IP), token ring, IEEE 802.11a/b/g/n/x, etc. The communicationnetwork 708 may include, without limitation, a direct interconnection,local area network (LAN), wide area network (WAN), wireless network(e.g., using Wireless Application Protocol), the Internet, etc. Usingthe network interface 707 and the communication network 708, thecomputer system 701 may communicate with devices 709 and 710. Thesedevices may include, without limitation, personal computer(s),server(s), fax machines, printers, scanners, various mobile devices suchas cellular telephones, smartphones (e.g., Apple iPhone, Blackberry,Android-based phones, etc.), tablet computers, eBook readers (AmazonKindle, Nook, etc.), laptop computers, notebooks, gaming consoles(Microsoft Xbox, Nintendo DS, Sony PlayStation, etc.), or the like. Insome embodiments, the computer system 701 may itself embody one or moreof these devices.

In some embodiments, the processor 702 may be disposed in communicationwith one or more memory devices (e.g., RAM 613, ROM 614, etc.) via astorage interface 712. The storage interface may connect to memorydevices including, without limitation, memory drives, removable discdrives, etc., employing connection protocols such as serial advancedtechnology attachment (SATA), integrated drive electronics (IDE),IEEE-1394, universal serial bus (USB), fiber channel, small computersystems interface (SCSI), etc. The memory drives may further include adrum, magnetic disc drive, magneto-optical drive, optical drive,redundant array of independent discs (RAID), solid-state memory devices,solid-state drives, etc. Variations of memory devices may be used forimplementing, for example, any databases utilized in this disclosure.

The memory devices may store a collection of program or databasecomponents, including, without limitation, an operating system 716, userinterface application 717, user/application data 718 (e.g., any datavariables or data records discussed in this disclosure), etc. Theoperating system 716 may facilitate resource management and operation ofthe computer system 701. Examples of operating systems include, withoutlimitation, Apple Macintosh OS X, Unix, Unix-like system distributions(e.g., Berkeley Software Distribution (BSD), FreeBSD, NetBSD, OpenBSD,etc.), Linux distributions (e.g., Red Hat, Ubuntu, Kubuntu, etc.), IBMOS/2, Microsoft Windows (XP, Vista/7/8, etc.), Apple iOS, GoogleAndroid, Blackberry OS, or the like. User interface 717 may facilitatedisplay, execution, interaction, manipulation, or operation of programcomponents through textual or graphical facilities. For example, userinterfaces may provide computer interaction interface elements on adisplay system operatively connected to the computer system 701, such ascursors, icons, check boxes, menus, scrollers, windows, widgets, etc.Graphical user interfaces (GUIs) may be employed, including, withoutlimitation, Apple Macintosh operating systems' Aqua, IBM OS/2, MicrosoftWindows (e.g., Aero, Metro, etc.), Unix X-Windows, web interfacelibraries (e.g., ActiveX, Java, Javascript, AJAX, HTML, Adobe Flash,etc.), or the like.

In some embodiments, computer system 701 may store user/application data618, such as the data, variables, records, etc. as described in thisdisclosure. Such databases may be implemented as fault-tolerant,relational, scalable, secure databases such as Oracle or Sybase.Alternatively, such databases may be implemented using standardized datastructures, such as an array, hash, linked list, structured text file(e.g., XML), table, or as hand-oriented databases (e.g., usingHandStore, Poet, Zope, etc.). Such databases may be consolidated ordistributed, sometimes among various computer systems discussed above.It is to be understood that the structure and operation of any computeror database component may be combined, consolidated, or distributed inany working combination.

Additionally, in some embodiments, the server, messaging andinstructions transmitted or received may emanate from hardware,including operating system, and program code (i.e., application code)residing in a cloud implementation. Further, it should be noted that oneor more of the systems and methods provided herein may be suitable forcloud-based implementation. For example, in some embodiments, some orall of the data used in the disclosed methods may be sourced from orstored on any cloud computing platform.

The written description describes the subject matter herein to enableany person skilled in the art to make and use the embodiments. The scopeof the subject matter embodiments is defined by the claims and mayinclude other modifications that occur to those skilled in the art. Suchother modifications are intended to be within the scope of the claims ifthey have similar elements that do not differ from the literal languageof the claims or if they include equivalent elements with insubstantialdifferences from the literal language of the claims.

Various embodiments disclosed herein provides a learning-based modellingof an emergent behavior of a complex system. The disclosed systemincludes a digital twin of the complex system and a digital twin of anenvironment of said complex system, and captures an interaction anddynamic behavior of agents of the digital twins. The agents of thedigital twins are modelled using learning-based models such as RL andgenetic algorithms that learns the behavior (i.e. actions and theiroutcomes) over a period of time. Hence, the agents (or actors) of thedigital twins are dynamic in nature. The actor-based bottom upsimulation approach is capable of producing sufficient insight foreffective decision making prior to implementation.

It is to be understood that the scope of the protection is extended tosuch a program and in addition to a computer-readable means having amessage therein; such computer-readable storage means containprogram-code means for implementation of one or more steps of themethod, when the program runs on a server or mobile device or anysuitable programmable device. The hardware device can be any kind ofdevice which can be programmed including e.g. any kind of computer likea server or a personal computer, or the like, or any combinationthereof. The device may also include means which could be e.g. hardwaremeans like e.g. an application-specific integrated circuit (ASIC), afield-programmable gate array (FPGA), or a combination of hardware andsoftware means, e.g. an ASIC and an FPGA, or at least one microprocessorand at least one memory with software processing components locatedtherein. Thus, the means can include both hardware means and softwaremeans. The method embodiments described herein could be implemented inhardware and software. The device may also include software means.Alternatively, the embodiments may be implemented on different hardwaredevices, e.g. using a plurality of CPUs.

The embodiments herein can comprise hardware and software elements. Theembodiments that are implemented in software include but are not limitedto, firmware, resident software, microcode, etc. The functions performedby various components described herein may be implemented in othercomponents or combinations of other components. For the purposes of thisdescription, a computer-usable or computer readable medium can be anyapparatus that can comprise, store, communicate, propagate, or transportthe program for use by or in connection with the instruction executionsystem, apparatus, or device.

The illustrated steps are set out to explain the exemplary embodimentsshown, and it should be anticipated that ongoing technologicaldevelopment will change the manner in which particular functions areperformed. These examples are presented herein for purposes ofillustration, and not limitation. Further, the boundaries of thefunctional building blocks have been arbitrarily defined herein for theconvenience of the description. Alternative boundaries can be defined solong as the specified functions and relationships thereof areappropriately performed. Alternatives (including equivalents,extensions, variations, deviations, etc., of those described herein)will be apparent to persons skilled in the relevant art(s) based on theteachings contained herein. Such alternatives fall within the scope ofthe disclosed embodiments. Also, the words “comprising,” “having,”“containing,” and “including,” and other similar forms are intended tobe equivalent in meaning and be open ended in that an item or itemsfollowing any one of these words is not meant to be an exhaustivelisting of such item or items, or meant to be limited to only the listeditem or items. It must also be noted that as used herein and in theappended claims, the singular forms “a,” “an,” and “the” include pluralreferences unless the context clearly dictates otherwise.

Furthermore, one or more computer-readable storage media may be utilizedin implementing embodiments consistent with the present disclosure. Acomputer-readable storage medium refers to any type of physical memoryon which information or data readable by a processor may be stored.Thus, a computer-readable storage medium may store instructions forexecution by one or more processors, including instructions for causingthe processor(s) to perform steps or stages consistent with theembodiments described herein. The term “computer-readable medium” shouldbe understood to include tangible items and exclude carrier waves andtransient signals, i.e., be non-transitory. Examples include randomaccess memory (RAM), read-only memory (ROM), volatile memory,nonvolatile memory, hard drives, CD ROMs, DVDs, flash drives, disks, andany other known physical storage media.

It is intended that the disclosure and examples be considered asexemplary only, with a true scope of disclosed embodiments beingindicated by the following claims.

What is claimed is:
 1. A processor-implemented method comprising:simulating, via one or more hardware processors, a first digital twin ofa complex system and a second digital twin of an environment associatedwith the complex system, the first digital twin comprising a first setof digitally configured dynamic agents and the second digital twincomprising a second set of digitally configured dynamic agents, each ofthe first set and the second set of digitally configured dynamic agentsdefined using one or more state variables, one or more characteristicvariables and a set of actions; receiving, via the one or more hardwareprocessors, a trigger at one or more digitally configured dynamic agentsfrom amongst the first set and the second set of digitally configureddynamic agents; computing, via the one or more hardware processors, acurrent value of the one or more state variables and the one or morecharacteristic variables associated with the one or more digitallyconfigured dynamic agents by accessing a system database; predicting,via the one or more hardware processors, a difference between thecurrent value and an expected value of the one or more state variablesand the one or more characteristics variables, the current valuecomputed using a multi criteria decision making technique, the expectedvalue obtained from one or more goals associated with the complexsystem, the one or more goals prestored in a knowledge repositoryassociated with the complex system, and wherein the one or more goalsare indicative of decision-making in response to the trigger in thecomplex system; defining, based on the difference between the currentvalue and the expected value of the one or more state variables and theone or more characteristic variables, a decision function for the one ormore digitally configured dynamic agents using a decision function, viathe one or more hardware processors, wherein the decision function isrealized using at least one of a Reinforcement learning (RL) andoptimization technique, and wherein the an observation is indicative ofoutcome of the decision function and ability to reach to the desiredstate in the future time; and simulating iteratively, via the one ormore hardware processors, the first and second set of digitallyconfigured dynamic agents based on the decision function in a pluralityof iterations until the difference between the current value andexpected value of the one or more state variables is determined to bewithin a predetermined threshold limit.
 2. The processor implementedmethod of claim 1, wherein an action form amongst the set of actions isdefined using a tuple comprising an event, the trigger, a computationfunction, and a resistance value, and wherein the computation functionis a function of the one or more state variables and the one or morecharacteristics variables, and wherein the computation function iscalculated using Multicriteria decision-making (MCDM) technique, andwherein the resistance comprises a threshold, wherein the threshold is afunction over the one or more characteristics variables.
 3. Theprocessor implemented method of claim 1, wherein each of the first setand the second set of digitally configured dynamic agents comprises anRL agent capable of observing action taken by the decision function andconsequence of the action during the iterative simulations.
 4. Theprocessor implemented method of claim 3, further comprising triggering amodification in the one or more characteristics variables of a digitallyconfigured dynamic agent from amongst the one or more digitallyconfigured dynamic agents based on a series of actions taken by thedigitally configured dynamic agent over a predefined period of time andobserving values in terms of the one or more state variables incognizance of the one or more goals of the digitally configured dynamicagent and actual state variable after the predefined period of time. 5.A system comprising: a memory storing instructions; one or morecommunication interfaces; and one or more hardware processors coupled tothe memory via the one or more communication interfaces, wherein the oneor more hardware processors are configured by the instructions to:simulate a first digital twin of a complex system and a second digitaltwin of an environment associated with the complex system, the firstdigital twin comprising a first set of digitally configured dynamicagents and the second digital twin comprising a second set of digitallyconfigured dynamic agents, each of the first set and the second set ofdigitally configured dynamic agents defined using one or more statevariables, one or more characteristic variables and a set of actions;receive a trigger at one or more digitally configured dynamic agentsfrom amongst the first set and the second set of digitally configureddynamic agents; compute a current value of the one or more statevariables and the one or more characteristic variables associated withthe one or more digitally configured dynamic agents by accessing asystem database; predict a difference between the current value and anexpected value of the one or more state variables and the one or morecharacteristics variables, the current value computed using multicriteria decision making technique, the expected value obtained from oneor more goals associated with the complex system, the one or more goalsprestored in a knowledge repository associated with the complex system,and wherein the one or more goals are indicative of decision-making inresponse to the trigger in the complex system; define, based on thedifference between the current value and the expected value of the oneor more state variables and the one or more characteristic variables, adecision function for the one or more digitally configured dynamicagents using a decision function, wherein the decision function isrealized using at least one of a Reinforcement learning (RL) andoptimization technique, and wherein the an observation is indicative ofoutcome of the decision function and ability to reach to the desiredstate in the future time; and simulate iteratively the first and secondset of digitally configured dynamic agents based on the decisionfunction in a plurality of iterations until the difference between thecurrent value and expected value of the one or more state variables isdetermined to be within a predetermined threshold limit.
 6. The systemof claim 5, wherein an action of the set of actions is defined using atuple comprising an event, the trigger, a computation function, and aresistance value, and wherein the computation function is a function ofthe one or more state variables and the one or more characteristicsvariables, and wherein the computation function is calculated usingMulticriteria decision-making (MCDM) technique, and wherein theresistance comprises a threshold, wherein the threshold is a functionover the one or more characteristics variables.
 7. The system of claim5, wherein each of the first set and the second set of digitallyconfigured dynamic agents comprises an reinforcement learning (RL) agentcapable of observing action taken by the decision function andconsequence of the action during the iterative simulations.
 8. Thesystem of claim 7, wherein the one or more hardware processors arefurther configured by the instructions to trigger a modification in theone or more characteristics variables of a digitally configured dynamicagent from amongst the one or more digitally configured dynamic agentsbased on a series of actions taken by the digitally configured dynamicagent over a predefined period of time and observing values in terms ofthe one or more state variables in cognizance of the one or more goalsof the digitally configured dynamic agent and actual state variableafter the predefined period of time.
 9. One or more non-transitorymachine readable information storage mediums comprising one or moreinstructions which when executed by one or more hardware processorscause: simulating, via one or more hardware processors, a first digitaltwin of a complex system and a second digital twin of an environmentassociated with the complex system, the first digital twin comprising afirst set of digitally configured dynamic agents and the second digitaltwin comprising a second set of digitally configured dynamic agents,each of the first set and the second set of digitally configured dynamicagents defined using one or more state variables, one or morecharacteristic variables and a set of actions; receiving, via the one ormore hardware processors, a trigger at one or more digitally configureddynamic agents from amongst the first set and the second set ofdigitally configured dynamic agents; computing, via the one or morehardware processors, a current value of the one or more state variablesand the one or more characteristic variables associated with the one ormore digitally configured dynamic agents by accessing a system database;predicting, via the one or more hardware processors, a differencebetween the current value and an expected value of the one or more statevariables and the one or more characteristics variables, the currentvalue computed using a multi criteria decision making technique, theexpected value obtained from one or more goals associated with thecomplex system, the one or more goals prestored in a knowledgerepository associated with the complex system, and wherein the one ormore goals are indicative of decision-making in response to the triggerin the complex system; defining, based on the difference between thecurrent value and the expected value of the one or more state variablesand the one or more characteristic variables, a decision function forthe one or more digitally configured dynamic agents using a decisionfunction, via the one or more hardware processors, wherein the decisionfunction is realized using at least one of a Reinforcement learning (RL)and optimization technique, and wherein the an observation is indicativeof outcome of the decision function and ability to reach to the desiredstate in the future time; and simulating iteratively, via the one ormore hardware processors, the first and second set of digitallyconfigured dynamic agents based on the decision function in a pluralityof iterations until the difference between the current value andexpected value of the one or more state variables is determined to bewithin a predetermined threshold limit.